Expectations

• One workshop per week

• One lecture per week

• One assignment per week

• One ‘capstone’ project

What to expect during this course

What is Reproducibility?

Introduction to Reproducible Research

• For Research to be reproducible both data and methods should be available.

• Applying the described methods to the data leads to the same results

Methods

• In theory, method availability ≠ code

• But with complex data and analyses - are methods of data collection enough?

Self-correcting Science?

• Science advances incrementally by identifying and rectifying errors over time

• Peer review: Critical evaluation of papers by experts maintain quality

• Independent studies either support or fail to replicate findings

Self-correcting Science?

• Publication bias: preference for positive results

• Pressure to publish

• Poor study designs and statistical issues

• Lack of transparency

Reproducibility trial:

246 biologists get different results from same data sets

Reproducibility Crisis

• The reproducibility crisis emerged when numerous studies, especially in fields like psychology, medicine, and biology, failed to be replicated by other researchers.

• High-profile replication attempts revealed that many published results could not be consistently reproduced, raising doubts about their validity.

Crisis as an Opportunity

• Recognition that no study should be considered ‘definitive’

• Empower lasting systemic change through increased transparency in research methods, data sharing and reporting

• Structural change in academic culture

Open Science

Open science is a global movement that aims to make scientific research and its outcomes freely accessible to everyone. By fostering practices like data sharing and preregistration, open science not only accelerates scientific progress but also strengthens trust in research findings.

UKRN

Tidy projects

/home/phil/Documents/paper
├── abstract.R
├── correlation.png
├── data.csv
├── data2.csv
├── fig1.png
├── figure 2 (copy).png
├── figure.png
├── figure1.png
├── figure10.png
├── partial data.csv
├── script.R
└── script_final.R

Organised projects

• README

• Documented

• Easy to code with

• All files are inside the root folder

R projects

Slugs

• A string of characters defining a file

What do you think are the contents of these files:

• data/raw/madrid_minimum-temperature.csv

• scripts/02_compute_mean-temperature.R

• analysis/01_madrid_minimum-temperature_descriptive-statistics.qmd

Name files

Come up with good names for these:

• a dataset of cats with columns for weight, length, tail length, fur colour(s), fur type and name.

• a script that downloads data from Spotify.

• a scripts that cleans up data.

• a scripts that fits a linear discriminant model and saves it to a file.

R projects and clean slates

• Use projects

• Check your code runs on blank slates

Quarto

Git

Git repository

Git collab

Forking

Renv

Benefits

Building Statistical Models

Populations and Samples

Population:

• The entire group you want to study (e.g., all humans, all mice in a lab).

• Studying the entire population is often impractical due to time, cost, or logistics.

Sample:

• A subset of the population, selected to make inferences about the whole.

• Must be representative to ensure accurate conclusions.

Descriptive Statistics

• Summarize data to highlight key features:

• Central Tendency: Where is the “center” of the data? (mean, median, mode)

• Spread: How variable are the data? (variance, standard deviation)

• Helps us understand the data before making inferences.

Central tendency

The central tendency of a series of observations is a measure of the “middle value”

The three most commonly reported measures of central tendency are the sample mean, median, and mode.

Mean

One of the simplest statistical models in biology is the mean

The mean

One of the simplest statistical models in biology is the mean

Symmetrical

If data is symmetrically distributed, the mean and median will be close, especially as n increases.

This is also know as a normal distribution

Variance & Standard Deviation

Variance:

The average of the squared differences from the mean.

\[s{^2}_{sample} = \frac{\sum(x - \bar{x})^2}{N -1}\] Higher variance = more spread.

Standard Deviation (SD):

• The square root of variance.

• Easier to interpret because it’s in the same units as the data.

Calculating variance

Standard Deviation

• Square root of sample variance

• A measure of dispersion of the sample

• Smaller SD = more values closer to mean, larger SD = greater data spread from mean

• variance:

\[ \sigma = \sqrt{\sum(x - \overline x)^2\over n - 1} \] ## N-1? {.smaller}

For a population the variance \(S_p^2\) is exactly the mean squared distance of the values from the population mean

\[ s{^2}_{pop} = \frac{\sum(x - \bar{x})^2}{N} \]

But this is a biased estimate for the population variance

• A biased sample variance will underestimate population variance

• n-1 (if you take a large enough sample size, will correct for this)

More here

Standard deviation - our example

Standard Deviation

Small \(s\) = data points are clustered near the mean

Large \(s\) = data points are widely dispersed around the mean

Why the Normal Distribution matters

Shape: Symmetrical, bell-shaped curve

• Described by just two parameters: mean (μ) and standard deviation (σ).

• Rule of Thumb: For normally distributed data:

  ~68% within 1 SD

  ~95% within 2 SDs

  ~99.7% within 3 SDs

Relevance: Many statistical tests assume data follows a normal distribution. This helps us calculate probabilities—like p-values—to test hypotheses.

Why the Normal Distribution matters

If we assume a normal distribution (or close enough), we can calculate the probability of observing any given value using just the mean and standard deviation.

\[ f(x) = \frac{1}{\sqrt{2\pi} \, \sigma} \exp\!\Biggl(-\frac{(x - \mu)^2}{2\,\sigma^2}\Biggr) \]

This has applications in hypothesis testing and building confidence intervals

Visualising a Distribution

Z-Scores and the Standard Normal Distribution

Problem: Different datasets have different means and standard deviations.

Solution: Standardization allows comparisons by converting any normal distribution into a standard normal distribution (mean = 0, SD = 1).

\[ Z = \frac{X - \mu}{\sigma} \]

• Z: How many standard deviations a value is from the mean.

• X: Observed value.

• μ: Population mean.

• σ: Population standard deviation.

Why do Z scores matter?

• The standard normal distribution allows us to calculate probabilities of observing extreme values.

• Example: If 𝑍 = 2, the observation is 2 standard deviations above the mean.

Using a Z-table, we can find the probability of getting a result at least this extreme.

This concept extends to p-values, which tell us how rare our data is under the null hypothesis.

What is a P-value?

Example

Imagine flipping a coin 100 times, expecting about 50 heads if it’s fair (the null hypothesis). If you observe 90 heads, the p-value tells you how likely it is to get such an extreme result just by chance.

Hypothesis formation

Null Hypothesis Probability Distribution

• The shaded areas in the tails represent extreme outcomes (typically the most unexpected 5% if using an \(\alpha\) = 0.05).

• If your observed data falls within these tails, it’s considered statistically significant, suggesting it’s unlikely to occur by random chance under the null hypothesis.

Impact of Sample Size on Distributions

• Key Point: Larger sample sizes reduce variability, making it easier to detect small differences as statistically significant.

• However, statistical significance doesn’t always mean practical importance—especially with large samples.

Hypothesis testing

lm(longevity ~ diet, data = mice)

Explanation: This linear model performs a t-test to assess if the mean lifespans differ significantly between diets.

Visualising the T Distribution

• The t-distribution shows where your observed mean difference falls relative to what’s expected under the null hypothesis.

• Confidence Intervals (CI): The red area marks the 95% CI. If zero falls outside this range, it suggests statistical significance.

# A tibble: 2 × 7
  term        estimate std.error statistic  p.value conf.low conf.high
  <chr>          <dbl>     <dbl>     <dbl>    <dbl>    <dbl>     <dbl>
1 (Intercept)     4.19     0.276     15.2  3.87e-13     3.62     4.77 
2 dietmice_b     -1.41     0.391     -3.60 1.60e- 3    -2.22    -0.595

Writing About Statistical Results

“Diets can change the longevity of mice (p = 0.0016).”

Common misundertandingsabout P values

A p-value is NOT:

• The probability that the null hypothesis is true.

• The probability your results occurred “by chance.”

• Proof of a meaningful or large effect.

What it IS:

• A measure of how surprising your data is under the assumption the null hypothesis is true.

Common misconceptions about P-values

Evidence of P-hacking

A non-significant p-value DOES NOT mean the null hypothesis is true

In reality, we can’t know for sure if a true mean difference exists.

For illustration: Assume we could know the true mean difference.

The figure shows:

Grey line: Expected data if the null hypothesis is true.

Black line: Expected data if the alternative hypothesis is true.

A p-value shows how surprising the data are if the null is true.

A low p-value is evidence against the null, not proof of the alternative.

Why a significant p-value does not mean the null hypothesis is false

What we can conclude, based on our data, is that we have observed an extreme outcome, that should be considered surprising. But such an outcome is not impossible when the null-hypothesis is true.

Why a significant p-value is not the same as an important effect

Additional resources

• Discovering Statistics - Andy Field

• Happy Git

• An Introduction to Generalized Linear Models - Dobson & Barnett

• An Introduction to Statistical Learning with Applications in R - James, Witten, Hastie & Tibshirani

• Mixed Effects Models and Extensions in Ecology with R - Zuur, et al.

• Ecological Statistics with contemporary theory and application

• The Big Book of R (https://www.bigbookofr.com/)

• British Ecological Society Guides to Better Science

*(SORTEE)